A Parallel Data Mining Architecture for Massive Data Sets

This paper discusses a parallel data mining architecture which provides the capability to mine massive data sets highly efficiently, scanning millions of rows of data per second. In this architecture the mining process is divided into two distinct components. A parallel server, Compaq’s Data Mining Server (DMS), provides a set of data mining primitives which are utilized by a data mining client, Syllogic’s DMT/MP, which implements the actual data mining algorithms. The parallel architecture and the primitives used to operate on the data will be discussed, and the mining algorithms’ use of these primitives. Performance figures will be presented for both the primitives and the high level mining algorithms.